Automatic method to delineate or categorize an electrocardiogram

ABSTRACT

A method for computerizing delineation and/or multi-label classification of an ECG signal, includes: applying a neural network to the ECG whereby labelling the ECG, and optionally displaying the labels according to time, optionally with the ECG signal.

FIELD OF INVENTION

The present invention relates to temporal signal analysis, preferablyECG analysis, using at least one neural network.

BACKGROUND OF INVENTION

The electrocardiogram (ECG) is a graphic representation of theelectrical activity of the heart. It is recorded from the body surfaceusing a number of electrodes placed in specific predefined areas. It isconsidered as a fundamental tool of clinical practice. It is a simple,non-invasive exam that can be performed by any health professional.Placing the electrodes is not considered as a medical procedure, yet insome countries, the prescription of the ECG by a doctor is essential forit to be performed. The ECG constitutes the first step in cardiovasculardiseases (CVD) diagnosis, and is used multiple times throughout the lifeof a CVD patient, CVD constitute the first global cause of death.

An ECG is composed of multiple temporal signals, called lead signals,such as the standard 12-lead ECG shown in FIG. 1. An ECG displaysrepeating patterns usually comprising a P-wave, a QRS complex and aT-wave, respectively corresponding to the depolarization of the atria,depolarization of the ventricles and repolarization of the ventricles.These waves and complex are shown in FIG. 2, which focuses on a coupleof beats in one lead signal.

The ECG allows for the detection of many anomalies, which often in turnpoint to specific CVD. It is estimated that about 150 measurableanomalies can be identified on ECG recordings today. However, withoutspecific expertise and/or regular training, only a small portion ofthese anomalies can be easily spotted. Unfortunately, today, it isestimated that only one third of ECGs are performed in settings wherecardiology expertise is readily available.

In order to make ECO interpretation simpler and assist non-specialists,two alternatives exist today, but neither fully satisfy the needs ofhealth professionals:

-   -   Telecardiology centers, where an interpretation of an ECG sent        by a non-specialist is delivered either by cardiologists or by        specialized ECG technicians. Their interpretations am of high        quality but are slow and expensive to obtain.    -   Prior art automated ECG interpretation softwares, which are        mainly developed by ECG device manufacturers. They provide low        quality interpretation (false alarms are very frequent) but        deliver them in seconds.

Prior art automated ECG interpretation softwares can provide two typesof information about an ECG signal:

-   -   the temporal locations of each wave, called its “delineation”,        and/or    -   a classification of the ECG as normal/abnormal or labeling its        anomalies.

Two main approaches are used for the delineation of ECG signals.

The first one is based on multiscale wavelet analysis. This approachlooks for wavelet coefficients reaching predefined thresholds atwell-chosen scales (Martinez et al, IEEE transactions on biomedicalengineering. Vol. 51, No. 4, April 2004, 570-581, Almeida et al., IEEEtransactions on biomedical engineering, Vol. 56, No. 8, August 2009, pp1996-2005, Boichat et al., Proceedings of Wearable and Implantable BodySensor Networks, 2009, pp 256-261, U.S. Pat. No. 8,903,479, 2014 Dec. 2,Zoicas et al.). The usual process is to look for QRS complexes, and thenlook for P waves on the signal before the complexes, and after them forT waves. This approach can only handle a single lead at a time,sometimes using projection to one artificial lead (US2014/0148714-2014-05-29, Mamaghanian et al). This computation is madevery unstable by the use of thresholds. The approach is also limited asit can neither deal with multiple P waves nor with “hidden” P waves. Ahidden P wave is a P wave which occurs during another wave or complex,such as for example during a T wave.

The second one is based on Hidden Markov Models (HMM). This machinelearning approach considers that the current state of the signal(whether a sample is either part of a QRS complex, a P wave, a T wave orno wave) is a hidden variable that one wants to recover (Coast et al.,IEEE transactions on biomedical engineering, Vol. 37, No. 9, September1990, pp 826-836, Hughes et al., Proceedings of Neural InformationProcessing Systems, 2004, pp 611-618, U.S. Pat. No. 8,332,017, 2012 Dec.11, Trassenko et al). To this end, a representation of the signal mustbe designed using handcrafted “features”, and a mathematical model mustbe fitted for each wave, based on these features. Based on a sufficientnumber of examples, the algorithms can learn to recognize each wave.This process can however be cumbersome since the feature design is notstraightforward, and the model, usually Gaussian, is not well adapted.Also, none of these works has considered the situation of hidden Pwaves.

As for anomalies and/or CVD detection, most algorithms use rules basedon temporal and morphological indicators computed using the delineation:PR, RR and QT intervals, QRS width, level of the ST segment, slope ofthe T wave, etc. . . . These rules such as the Minnesota Code (Prineaset al., Springer, ISBN 978-1-84882.777-6, 2009) were written bycardiologists. However, they do not reflect the way the cardiologistsanalyze the ECGs and are crude simplifications. Algorithms such as theGlasgow University Algorithm are based on such principles (Statement ofValidation and Accuracy for the Glasgow 12-Lead ECG Analysis Program,Physio Control, 2009).

More advanced methods use learning algorithms, and are built using adiagnosis and an adequate representation for each ECG they learn from;however, in these methods, once again, it is necessary to seek arepresentation of the raw data into a space that preserves theinvariance and stability properties. Indeed, an ECG signal variessignificantly from one patient to another. It is therefore extremelydifficult for an algorithm to learn how to discriminate differentdiseases by simply comparing raw data. A representation whichdrastically limits this interpatient variability while preserving theinvariance within the same disease class must be chosen.

In order to solve the above-mentioned issues, the Applicant turned toarchitectures called neural network. Such architectures have beenintensively studied in the field of imaging (Russakovsky et al.,arXiv:1409.0575v3, 30 Jan. 2015), but limitations arose when, veryrecently, the first scientific teams attempted to apply them to ECGs(Zheng et al., Web-Age Information Management, 2014, Vol. 8485, pp298-310, Jin and Dong, Science China Press, Vol. 45, No 3, 2015, pp398-416). Indeed, these prior arts only limit the classification to anattempt to identify normal ECG versus abnormal ECG, or to perform abeat-to-beat analysis. The beat-to-beat analysis adds a preprocessingstep while reducing the ability of the neural network to learn someanomalies: rhythm disorders, for example, cannot be identified from theobservation of a single beat. In fact, these algorithms only considersingle-label classification whereas multi-label classification isessential in ECG interpretation, since one ECG can present severalanomalies.

Thus, there is a need for computerized algorithms able to analyze ECGthat can:

-   -   carry out the analysis without constraints from the ECG        recording duration;    -   carry out the analysis without the need for beat-by-beat        processing, or feature extraction;    -   obtain the delineation of the signal, including identification        of hidden P waves;    -   provide a multi-label classification directly from a full ECG,        possibly exhibiting multiple labels;    -   be fast, stable and reliable.

SUMMARY

To address the above issues in ECG analyses, the Applicant developed twotechniques based on convolutional neural networks.

-   -   A fully convolutional neural network which first gives a dense        prediction of the probability of presence of each wave on each        time stamp of the ECG, then post-processes the signal to produce        its delineation. This novel approach of the delineation, using        convolutional networks, allows the processing of ECGs of any        duration, analyzing all types of waves in the same way, without        being constrained by their positions.    -   A recurrent convolutional neural network which predicts directly        multiple labels on the whole ECG signal. This structure allows        the processing of an ECG of any duration, and takes into account        the time dynamic in its analysis. It results in a fixed format        output (multi-labels).

Thus, the present invention relates to a method for computerizing thedelineation of an ECG signal, comprising: applying a fully convolutionalneural network NN1 to said ECG, whereby the fully convolutional neuralnetwork NN1 reads each time point of the ECG signal, analyzes temporallyeach time point of the ECG signal, assigns to each time point of the ECGa score for at least the following: P-wave, QRS complex, T-wave, andthen, optionally and whenever necessary, displaying the scores accordingto time, optionally with the ECG signal.

According to an embodiment, the method further comprises a pre-treatmentstep, wherein the pre-treatment comprises denoising and removing thebaseline of the ECG signal as well as expressing it at a chosenfrequency prior to the application of NN1.

According to an embodiment, the method further comprises apost-treatment step, so as to produce the time points of the beginningand the end of each wave in the ECG signals, called the onsets andoffsets.

The invention also comprises a software comprising a trained neuralnetwork for delineation of an ECG signal. The invention also comprises acomputer device comprising a software implementing a method fordelineation of an ECG signal, comprising applying a fully convolutionalneural network NN1 to said ECG, as described above.

This invention also includes a method for computerizing multi-labelclassification of an ECG signal, comprising applying a convolutionalneural network NN2 to said ECG, whereby the recurrent neural network NN2reads each time point of the ECG signal, analyzes each time point of thesignal, computes scores for each anomaly and allots to an ECG at leastone disease-related label, and then, optionally, displaying the labels,preferably in the form of a list of detected anomalies.

According to an embodiment, the method further comprises a pre-treatmentstep, wherein the pre-treatment comprises denoising and removing thebaseline of the ECG signal as well as expressing it at a chosenfrequency prior to the application of NN2.

According to an embodiment, the method further comprises apost-treatment step, so as to produce the onset and offset of each wavein the ECG signal.

The invention also comprises a software comprising a trained neuralnetwork for multi-label classification of an ECG signal. The inventionalso comprises a computer device comprising a software implementing amethod for multi-label classification of an ECG signal, comprisingapplying a recurrent neural network NN2 to said ECG, as described above.

Furthermore, the invention also includes a method for computerizingdelineation and multi-label classification of an ECG signal, comprisingapplying a trained recurrent neural network NN3 to said ECG, whereby therecurrent neural network NN3 reads each time point of the ECG signal,analyzes temporally each time point of the signal, assigns a score forat least the following: P-wave, QRS complex, T-wave, no wave; computesscores for each anomaly, and then, optionally, displaying the intervallabels according to time and the anomaly scores, preferably in the formof a list of detected, optionally with the ECG signal.

According to an embodiment, the method further comprises a pre-treatmentstep, wherein the pre-treatment comprises denoising and removing thebaseline of the ECG signal as well as expressing it at a chosenfrequency prior to the application of NN3.

According to an embodiment, the method further comprises apost-treatment step, so as to produce the onset and offset of each wavein the ECG signal.

The invention also comprises a software comprising a trained neuralnetwork for delineation and multi-label classification of an ECG signal.The invention also comprises a computer device comprising a softwareimplementing a method for delineation and multi-label classification ofan ECO signal, comprising applying a neural network NN3 to said ECG, asdescribed above.

DETAILED DESCRIPTION

The present invention relates to temporal signal analysis, preferablyECG analysis, using at least one convolutional neural network.

The framework used here is the one of supervised learning. The aim ofsupervised learning is to predict an output vector Y from an inputvector X. In the Applicant embodiment, X is an ECG (a multivariatesignal) as a matrix of size m×n. As for Y, in the Applicant embodiment,it can be:

-   -   the delineation, providing a score for each sample of X to be        part of one of the different waves as a matrix of size p×n;    -   the scores for each anomaly as a vector of size q;    -   the set composed of both the delineation and the vector of        scores.

The problem of supervised learning can also be stated as follows:designing a function f such that for any input X, f(X)≈Y. To this end,the function f is parameterized, and these parameters are “learned”(parameters are optimized with regards to an objective loss function,for example, by means of a gradient descent (Bishop, Pattern Recognitionand Machine Learning, Springer, 2006, ISBN-10: 0-387-31073-8).

A neural network is a particular type of function f, aiming at mimickingthe way biological neurons work. One of the most basic and earliestneural network is the perceptron (Rosenblatt, Psychological Review, Vol.65, No. 6, 1958, pp 386-408). From the input X, it computes linearcombinations (i.e. weighted sums) of the elements of X through amultiplication with a matrix W, adds an offset b, and then applies anon-linear function σ, such as for instance a sigmoid, on every elementof the output:

f(X)=σ(WX+B)

The parameters which are learned in a perceptron are both W and B. Inpractice, more general neural networks are just compositions ofperceptrons:

f(X)=σ_(n)(W _(n) . . . σ_(n)(W ₁ X+B ₁)+B _(n))

The output of a perceptron can be sent as input to another one. Theinput, the final output, and the intermediate states are called layers.The intermediate ones are more specifically called hidden layers, sinceonly the input and the final output are observed. For instance, a neuralnetwork with one hidden layer can be written as:

f(X)=σ₂(W ₂σ₁(W ₁ X+B ₁)+B ₂)

Such a network is shown in a graphic form as an example in FIG. 3. Thevector X enters the network as the input layer, each element of thehidden layer is then computed from linear combinations of all elementsof X (hence all the links), and the element of the output layer are thencomputed from linear combinations of all elements of the hidden layer.

It has been shown that neural networks in their general form are able toapproximate all kinds oft functions (Cybenko, Math, Control SignalsSystems, Vol. 2, 1989, pp 303-314). The term “deep learning” is usedwhen a neural network is composed of many layers (though the thresholdis not perfectly defined, it can be set to about ten). This field arosemostly in the last decade, thanks to recent advances in algorithms andin computation power.

Convolutional neural networks are a particular type of neural networks,where one or more of the matrices W_(i) which are learned do not encodea full linear combination of the input elements, but the same locallinear combination at all the elements of a structured signal such asfor example an image or, in this specific context, an ECG, through aconvolution (Fukushima, Biol. Cybernetics, Vol. 36, 1980, pp 193-202,LeCun et al., Neural Computation, Vol. 1, 1989, pp 541.551). Anillustration of a convolutional neural network is shown in FIG. 6. Mostconvolutional neural networks implement a few convolutional layers andthen standard layers so as to provide a classification. A network whichonly contains convolutional networks is called a fully convolutionalneural network. Finally, a recurrent convolutional neural network is anetwork composed of two sub-networks: a convolutional neural networkwhich extracts features and is computed at all time points of the ECG,and a neural network on top of it which aggregates in time the outputsof the convolutional neural network. An illustration of a recurrentconvolutional neural network is provided in FIG. 7.

As mentioned above, an ECG is represented as a matrix of real numbers,of size m×n. The constant m is the number of leads, typically 12, thoughnetworks could be taught to process ECG with any number of leads. Thenumber of samples n provides the duration of the ECG n/f, with f beingthe sampling frequency of the ECG. A network is trained for a givenfrequency, such as for example 250 Hz or 500 Hz or 1000 Hz, though anyfrequency could be used. A same network can however process ECG of anylength n, thanks to the fact that it is fully convolutional in theembodiment of the delineation, or thanks to the use of a recurrentneural network in the embodiment of the anomaly network.

In both the delineation and the multi-label classification embodiment s,networks are expressed using open softwares such as for example Theano.Caffe or Torch. These tools provide functions for computing theoutput(s) of the networks and for updating their parameters throughgradient descent. The exact structure of the network is not extremelyimportant as long as they are deep structures: fully convolutional inthe situation of the delineation network (Long et al., Proceedings ofComputer Vision and Pattern Recognition, 2015, pp 3431-3440), andconvolutional (Krizhevsk et al., Proceedings of Neural InformationProcessing Systems, 2012, pp 1097-1105), potentially recurrent in thesituation of the multi-label classification network (Donahue et al.,arXiv:1411.4389v3, 17 Feb. 2015 and Mnih et al., arXiv:1406.6247v1, 24Jun. 2014). The 2D convolutional layers which were used on images arethen easily converted into 1D convolutional layers in order to processECO signals.

This invention also pertains to a method for manufacturing a neuralnetwork for delineation of an ECG, by training it.

The training phase of the neural networks in the embodiment ofdelineation consists in the following steps:

-   -   taking one ECG from a dataset containing ECGs and their known        delineation; the ECG being expressed as a matrix of size m×n        with m fixed and at a predefined frequency;    -   expressing the delineation of this ECG under the form of a        matrix y of size p×n where p is the number of annotated types of        wave; typically p=3, so as to identify P waves, QRS complexes,        and T waves; annotations are usually expressed as lists of wave        with their start and end points such as for example: (P, 1.2s,        1.3s). (QRS 1.4s 1.7s), (T, 1.7, 2.1), (P, 2.2, 2.3); in this        example, the first row of y, corresponding to P waves, will be 1        for samples corresponding to times between 1.2 and 1.3s, and        between 2.2 and 2.4s, and 0 otherwise; row 2 will correspond to        QRS complexes and row 3 to T waves;    -   computing the output of the network for this ECG;    -   modifying the parameters of the network so as to decrease a cost        function comparing the known delineation and the output of the        network; a cross-entropy error function is used so as to allow        for multi-labeling (allowing for multiple waves at a given        instant); this minimization can be done though a gradient step    -   repeating steps 1 to 4 at least once for each ECG of the        dataset;    -   recovering the neural network.

This invention also provides a method for manufacturing a neural networkfor the categorization of an ECG signal, by training it.

In a multi-label classification, the manufacturing/training processincludes the following steps:

-   -   taking one ECG from a dataset containing ECGs and their known        anomaly labels; the ECG must be expressed as a matrix of size        m×n with m fixed and at a predefined frequency;    -   expressing the anomalies as a vector of size q, with q the        number of anomalies to identify; this vector could be [0; 1; 0;        0; 1; 0; 0; 0] for q=8; a 1 is set in the vector at the index        corresponding to the anomalies which are present: in the above        example, the ECG exhibits two anomalies;    -   computing the output of the network for this ECG;    -   modifying the parameters of the network so as to decrease a cost        function comparing the known label vector and the output of the        network; a cross-entropy error function is used so as to allow        for multi-labeling (allowing for multiple anomalies for an ECG);        this minimization can be done though a gradient step;    -   repeating steps 1 to 4 at least once for each ECG of the        dataset;    -   recovering the neural network.

This invention also provides a method for manufacturing a neural networkfor both the delineation and the categorization of an ECG signal, bytraining it.

In the embodiment of the combination of delineation with multi-labelclassification, the manufacturing process includes the following steps:

-   -   taking one ECG from a dataset containing ECGs and their known        anomaly labels; the ECG must be expressed as a matrix of size        m×n with m fixed and at a predefined frequency;    -   expressing the anomalies as a vector of size q, with q the        number of anomalies to identify; this vector could be [0; 1; 0;        0; 1; 0; 0; 0] for q=8; a 1 is set in the vector at the index        corresponding to the anomalies which are present: in the above        example, the ECG exhibits two anomalies;    -   expressing the delineation of this ECG under the form of a        matrix Y of size p×n where p is the number of waves to identify;        typically p=3, so as to identify P waves, QRS waves, and T        waves; annotations are usually expressed as lists wave type with        their start and end points such as for example: (P, 1.2s, 1.3s),        (QRS 1.4s 1.7s), (T, 1.7, 2.1), (P, 2.2, 2.3); in this example,        the first row of Y, corresponding to P waves, will be 1 for        samples corresponding to times between 1.2 and 1.3s, and between        2.2 and 2.4s, and 0 otherwise; row 2 will correspond to QRS        complexes and row 3 to T waves;    -   computing both outputs of the network for this ECG;    -   modifying the parameters of the network so as to decrease the        sum of a cost function comparing the known label vector and one        of the output of the network, and a cost function comparing the        delineation and the other output; cross-entropy error functions        are used to allow for multi-labeling (allowing for multiple        anomalies for an ECG as well as multiple waves at any time        point); this minimization can be done though a gradient step;    -   repeating steps 1 to 4 at least once for each ECG of the        dataset;    -   recovering the neural network.

This invention also pertains to a method and a device for delineation ofan ECG signal, implementing a fully convolutional neural network trainedfor delineation of an ECG signal as described above.

As a basis, it shall be understood that the ECG is expressed as a matrixX of size m×n at the frequency used for training the networks. The ECGis used as input of the trained neural network.

The neural network then reads each time point of the ECG signal,analyzes spatio-temporally each time point of the ECG signal, assigns atemporal interval score to anyone of at least the following: P-wave, QRScomplex, T-wave. It then recovers the output of the neural network, as amatrix Y of size p×n. An example is shown in FIG. 4: the first signalshows one of the leads of the ECG (to help the visualization), thefollowing 3 signals are the outputs of the network, providing scores forP waves, QRS waves and T waves. As it can be seen, these scores aresynchronized with the appearance of the aforementioned waves in thesignal.

In a preferred embodiment, the neural network provides scores at eachtime point as a matrix Y, and a post-processing allows the allocation ofeach time point to none, single, or several waves, and provides theonset and offset of each of the identified waves. For instance, a samplecan be affected to the waves for which the score on the correspondingrow of Y is larger than 0.5. This provides a delineation sequence oftype (P, 1.2s, 1.3s), (QRS 1.4s 1.7s), (T, 1.7s, 2.1s), (P, 2.2s, 2.3s),as recorded in the annotations.

In an embodiment, finally, the labels may be optionally displayedaccording to time, optionally with the ECG signal.

This invention also pertains to a method and a device for multi-labelclassification of an ECG signal, implementing Long-term RecurrentConvolutional Networks (LRCN, (Donahue et al., arXiv:1411.4389v3, 17Feb. 2015). These neural networks are trained for multi-labelclassification of an ECG signal as described above.

As a basis, it shall be understood that the ECG is expressed as a matrixof size m×n at the frequency used for training the networks. Then, theECG is used as input of the trained neural network.

The neural network then reads each time point of the ECG signal,analyzes temporally each time point of the ECG signal, computes a scorefor each anomaly, recovers the output of the neural network, andoptionally displays the labels.

In a preferred embodiment, the neural network recovers the output as avector of size q. This vector contains scores for the presence of eachanomaly.

According to an embodiment, the neural network displays the list offound anomalies as the elements for which the score in the vector arehigher than a predefined threshold, typically 0.5.

This invention also pertains to a method and a device for delineationand multi-label classification of an ECG signal, implementing a neuralnetwork trained for delineation and multi-label classification of an ECGsignal as described above.

As a basis, it shall be understood that the ECG is expressed as a matrixof size m×n at the frequency used for training the networks. Then, theECG is used as input of the trained neural network.

The neural network then reads each time point of the ECG signal,analyzes temporally each time point, assigns a temporal score to all ofthe following at least: P-wave, QRS complex, T-wave. It then computes ascore for each anomaly, recovers both the outputs of the neural network:the first as a matrix y of size p×n, providing scores for at least Pwaves, QRS waves and T waves; and the second as a vector of size q, saidvector containing scores for the presence of each anomaly.

In a preferred embodiment, a post-processing of the delineation outputallows to affect each time point to none, single, or several waves, andprovides the onset and offset of each of the identified waves. Forinstance, a sample can be affected to the waves for which the score onthe corresponding row of Y is larger than 0.5. This provides adelineation sequence of type (P, 1.2s, 1.3s), (QRS 1.4s 1.7s), (T, 1.7s,2.1s), (P, 2.2s, 2.3s), as recorded in the annotations.

According to an embodiment, the neural network displays the list offound anomalies as the elements for which the score in the vector arehigher than a predefined threshold, typically 0.5, as well as thedelineation, optionally with the ECG signal.

The invention also comprises a computer device implemented softwarecomprising a trained neural network for delineation of an ECG signal.The invention also comprises a device, such as f r example a cloudserver, a commercial ECG device, a mobile phone or a tablet, comprisinga software implementing a method for delineation, multi-labelclassification or both, of an ECG signal, as described above.

According to an embodiment of the invention, a step to prepare thesignal and create input variables for classification is further carriedout (“pre-treatment”). The purpose of this pre-treatment is to removethe disturbing elements of the signal such as for example noise andbaseline, low frequency signal due to respiration and patient motion, inorder to facilitate classification. For noise filtering, a multivariateapproach functional analysis proposed by (Pigoli and Sangalli,Computational Statistics and Data Analysis, vol. 56, 2012, pp 1482-1498)can be used. The low frequencies of the signal corresponding to thepatient's movements may be removed using median filtering as proposed by(Kaur et al., Proceedings published by International Journal of ComputerApplications, 2011, pp 30-36).

According to an embodiment of the invention, a post-treatment step isadded, so as to produce the onset and offset of each wave in the ECGsignal.

This invention brings to the art a number of advantages, some of thembeing described below:

-   -   The input of the networks are one or multilead ECG signals with        variable length, possibly preprocessed so as to remove noise and        baseline wandering due to patients movements, and express the        signal at a chosen frequency.    -   The output of a classification network is a vector of scores for        anomalies. These are not classification scores since one ECG can        present several anomalies. For example, the output of such        network could be a vector [0.98; 0.89; 0.00; . . . ] with the        corresponding labels for each element of the vector (Right        Bundle Branch Bloc; Atrial Fibrillation; Normal ECG; . . . ).        Scores are given between a scale of [0, 1] and the example        output vectors therefore indicates a right bundle branch block        and atrial fibrillations. A recurrent neural network        architecture can be added on the top of the convolutional        network (Donahue et al., arXiv:1411.4389v3, 17 Feb. 2015 and        Mnih et al., arXiv:1406.6247v1, 24 Jun. 2014). In this way, the        convolution network acts as a pattern detector whose output will        be aggregated in time by the recurrent network.    -   The output of the delineation network is a set of signals        spanning the length of the input ECG signal, providing the score        for being in P wave, a QRS complex, a T wave and potentially        other types of waves such as for example premature ventricular        complexes, flutter waves, or U waves. An example of output        signals is provided in FIG. 5.    -   The delineation network is not limited to recovering at most one        wave at each time point and therefore can identify several waves        at any time point, such as for instance hidden P waves,    -   No works applying convolutional networks to the delineation have        been made so far.

The underlying structure of the networks is not fundamental as long asit is a recurrent network for the multi-label classification network anda fully convolutional network for the delineation network. One can use astructure such as RLCN (Donahue et al., arXiv:1411.4389v3, 17 Feb. 2015and Mnih et al., arXiv:1406.6247v1, 24 Jun. 2014) for classification anda network similar as the one in (Long et al., Proceedings of ComputerVision and Pattern Recognition, 2015, pp 3431-3440) for delineation. Inboth embodiments, convolutional layers must be modified as 1Dconvolutions instead of 2D convolutions.

A hybrid network, sharing the first convolutional layers and divergingso as to provide both the delineation as one output, and the multi-labelclassification as another output is also used. This combination has theadvantage of being able to produce a multi-label classification helpedby the identification of the ECG waves.

EXAMPLES

The present invention is further illustrated by the following examples.

Example 1: Training for Delineation

This training was performed on 630 ECGs and the network evaluated onabout 900 beats from 70 different patients which were not used for thetraining phase. The false positive rate was 0.3% for QRS complexes and Twaves detection and their false negative rate was 0.2%, They wererespectively 5.4% and 4.2% for P waves. The precision of the wave onsets(beginnings) and offsets (ends) are detailed below:

Point Standard Deviation (ms) Bias (ms) P wave start 11.2 −1.6 P waveend 11.2 −4.0 QRS complex start 6.3 −2.1 QRS complex end 3.9 −1.1 T waveend 16.1 −4.6

Compared with state-of-the art algorithms, the precision was improved.In FIG. 5 for instance, the ECG exhibits an atrioventricular block whichmeans that the P waves and the QRS complexes are completely decoupled.In this example, the P waves are regular and QRS complexes happen atrandom times. One can observe in this example that the algorithmcorrectly found two P waves between the first QRS complex and the secondQRS complex, while most algorithms would not be able to find them sincethey look for only one P wave before each complex. The last P wave alsostarts before the end of the last T wave, adding complexity. Otheralgorithms would not have been able to find this hidden wave.

Example 2: Training for Multi-Label Classification

A network has been trained using about 85,000 ECGs and has beenevaluated on a Dataset including 20,000 patients which were not used inthe training phase. For the evaluation purpose, the multi-labelclassification obtained was simplified to normal (no anomaly)/abnormalif need be. The results in terms of accuracy, specificity andsensitivity were the following:

Accuracy Specificity Sensitivity 0.91 0.88 0.92

A graphical representation of how a standard multi-label is used on ECGsis displayed in FIG. 6. The ECG is given as input to the network, whichaggregates the information locally and then combines it layer by layerto produce a high-level multi-label classification of the ECG, in thisexample correctly recognizing atrial fibrillations and a right bundlebranch block. Such networks however take a fixed size as input and theprocess must be reproduced at different locations so as to analyze thewhole signal. FIG. 7 is an example of a graphic representation of arecurrent neural network which overcomes this issue. This type ofnetwork is made from a standard convolutional network computed at allpossible locations of the signal, and on top of which comes anothernetwork layer which aggregates the information. In this example, thenetwork correctly recognizes a premature ventricular complex (PVC, thefifth and largest beat) in the first part of the signal while the secondpan of the signal is considered as normal. The aggregated output istherefore PVC since this ECG has an anomaly and cannot therefore beconsidered as normal.

Example 3

In another embodiment, the applicant combines features described abovein examples 1 and 2. Such combination enables to combine the advantagesof both networks in a unique network, providing similar results for boththe delineations and the multi-label classifications.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a photo of an ECG.

FIG. 2 is a schematic representation of a normal EGG, with the P wave,the QRS complex/wave comprising the Q, R, S and J points, and the Twave,

FIG. 3 is an example of structure for a basic neural network with noconvolutional layer.

FIG. 4 is an example of the output of the delineation network on anormal ECG.

FIG. 5 is an example of the output of the delineation network on an ECGwith hidden P waves (high degree atrioventricular block).

FIG. 6 models the way a standard multi-label convolutional networkworks.

FIG. 7 models the way a multi-label recurrent convolutional networkworks.

1-4. (canceled)
 5. A computerized-method for classification of anelectrocardiogram (ECG) signal obtained from a patient using a neuralnetwork, the method comprising: training the neural network with adataset of pre-characterized ECG signals to generate a trained neuralnetwork; receiving the ECG signal sampled at a plurality of time points;analyzing the ECG signal at the plurality of time points using thetrained neural network; computing scores based on the analyzed ECGsignal to detect anomalies; and assigning labels to the ECG signal basedon the scores related to the detected anomalies.
 6. Thecomputerized-method of claim 5, further comprising denoising andremoving a baseline of the ECG signal.
 7. The computerized-method ofclaim 5, further comprising determining a location of at least oneanomaly in the EGG signal.
 8. The computerized-method of claim 5,wherein assigning labels to the ECG signal comprises assigning multiplelabels for at least one time point of the plurality of time point. 9-12.(canceled)
 13. The computerized-method of claim 5, further comprisingexpressing the ECG signal at a selected frequency.
 14. Thecomputerized-method of claim 5, further comprising displaying the scoresrelated to the detected anomalies.
 15. The computerized-method of claim5, further comprising displaying the labels and the ECG signal.
 16. Thecomputerized-method of claim 5, wherein assigning labels to the ECGsignal related to the detected anomalies is assigning labels of atrialfibrillation to the ECG signal.
 17. The computerized-method of claim 5,wherein: the neural network expresses the ECG signal as a matrix of sizem×n with m being a number of leads fixed and at a predefined frequencyand n being a number of samples.
 18. The computerized-method of claim 5,further comprising expressing the detected anomalies as a vector of sizeq, with q being a number of anomalies to identify.
 19. Thecomputerized-method of claim 5, wherein computing scores based on theanalyzed ECG signal to detect anomalies comprises computing scores ateach one of the plurality of time points.
 20. A system forclassification of an electrocardiogram (ECG) signal obtained from apatient using a neural network, the system comprising at least oneserver to: train the neural network with a dataset of pre-characterizedECG signals to generate a trained neural network; receive the ECG signalsampled at a plurality of time points; analyze the ECG signal at theplurality of time points using the trained neural network; computescores based on the analyzed ECG signal to detect anomalies; and assignlabels to the ECG signal based on the scores related to the detectedanomalies.
 21. The system of claim 20, wherein the at least one serveris configured to denoise and remove a baseline of the ECG signal. 22.The system of claim 20, wherein the at least one server is configured todetermine a location of at least one anomaly in the ECG signal.
 23. Thesystem of claim 20, wherein the at least one server is configured toassign multiple labels for at least one time point of the plurality oftime points.
 24. The system of claim 20, wherein the at least one serveris configured to express the ECG signal at a selected frequency.
 25. Thesystem of claim 20, wherein the at least one server is configured togenerate information indicative of the scores related to the detectedanomalies for display.
 26. The system of claim 20, wherein the at leastone server is configured to generate information indicative of thelabels and the ECG signal for display.
 27. The system of claim 20,wherein the at least one server is configured to assign labels of atrialfibrillation to the ECG signal.
 28. The system of claim 20, wherein theat least one server is configured to express the ECG signal as a matrixof size m×n with m fixed and at a predefined frequency.
 29. The systemof claim 20, wherein the at least one server is configured to expressthe one or more anomalies as a vector of size q, with q being a numberof anomalies to identify.
 30. The system of claim 20, wherein the atleast one server is configured to compute scores based on the analyzedECG signal at each one of the plurality of time points.
 31. A programmedroutine comprising instructions that, when executed by at least oneprocessor, cause the at least one processor to: train the neural networkwith a dataset of pre-characterized ECG signals to generate a trainedneural network; receive the ECG signal sampled at a plurality of timepoints; analyze the ECG signal at the plurality of time points using thetrained neural network; compute scores based on the analyzed ECG signalto detect anomalies; and assign labels to the ECG signal based on thescores related to the detected anomalies.
 32. The programmed routine ofclaim 31, further comprising instructions that, when executed by the atleast one processor, cause the at least one processor to denoise andremove a baseline of the ECG signal.
 33. The programmed routine of claim31, further comprising instructions that, when executed by the at leastone processor, cause the at least one processor to determine a locationof at least one anomaly in the ECG signal.
 34. The programmed routine ofclaim 31, further comprising instructions that, when executed by the atleast one processor, cause the at least one processor to assign multiplelabels for at least one time point of the plurality of time points. 35.The programmed routine of claim 31, further comprising instructionsthat, when executed by the at least one processor, cause the at leastone processor to compute scores based on the analyzed ECG signal at eachone of the plurality of time points.